Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use ASSET_n to download guest assets for virt tests #21123

Merged
merged 2 commits into from
Feb 13, 2025
Merged

Use ASSET_n to download guest assets for virt tests #21123

merged 2 commits into from
Feb 13, 2025

Conversation

Julie-CAO
Copy link
Contributor

@Julie-CAO Julie-CAO commented Feb 6, 2025 •
edited
Loading

Downloading assets from OSD require authentication to achieve CC compliance, refer to https://app.slack.com/client/T02863RC2AC/C02CANHLANP

Related ticket: https://progress.opensuse.org/issues/176589

Solution: Use ASSET_n to download guest assets.

Add ASSET_n0 to indicate the path of vm xml file, and ASSET_n1 for the vm disk file. fg.

ASSET_10=guest_%GUEST_LIST%-fv-def-net_on-host_%DISTRI%-%VERSION%_build%BUILD%_%SYSTEM_ROLE%_%ARCH%.xml 

ASSET_11=guest_%GUEST_LIST%-fv-def-net_on-host_%DISTRI%-%VERSION%_build%BUILD%_%SYSTEM_ROLE%_%ARCH%.disk

Advantages:

  • No authetication is required.
  • It is a more typical use of downloading files from OSD repo. It is safer and faster.
  • Don't start the test if there is no guest assets uploaded by its dependency tests.

Disadvantages:

  • Need add 2 or 4 openqa settings "ASSET_n" to a few tests which need download guest assets.
  • Don't start the test even if there are part of guest assets. ie. for a xen test, if there is only a pv asset, while missing a fv asset, the test would not start.
  • In the case you would not like to use the default value of GUEST_LIST , fg. to test a pv test only, you would have to re-set ASSET_10/11 accordingly.

Verification run:
a_chained_sriov_test
prj4_on_xen //pv & fv guest
prj4_guest_upgrade_sles15sp6_on_developing-xen //pv guest only
prj3

@alice-suse @waynechen55 @guoxuguang @nanzhg and others, welcome review!

@Julie-CAO Julie-CAO force-pushed the julie/no_check_download branch from d4187bc to 8758d29 Compare February 6, 2025 09:04
@RoyCai7
Copy link
Contributor

RoyCai7 commented Feb 7, 2025

LGTM.

@alice-suse
Copy link
Contributor

@Julie-CAO Hi Julie, yes , downloading assets from OSD require authentication, but still allowed. Can we add authentication related code instead?

@Julie-CAO
Copy link
Contributor Author

Julie-CAO commented Feb 7, 2025 •
edited
Loading

@Julie-CAO Hi Julie, yes , downloading assets from OSD require authentication, but still allowed. Can we add authentication related code instead?

I have not had a good idea about authentication. "curl" or "wget" with someone's username and API secrete? In this case, 3 more openqa settings are required and the automation would be complex.

@alice-suse
Copy link
Contributor

alice-suse commented Feb 7, 2025 •
edited by Julie-CAO
Loading

@Julie-CAO Hi Julie, yes , downloading assets from OSD require authentication, but still allowed. Can we add authentication related code instead?

I have not had a good idea about authentication. "curl" or "wget" with someone's username and API secrete? In this case, 3 more openqa settings are required and the automation would be complex.

Suggest to ask in https://suse.slack.com/archives/C02CANHLANP/p1737385075051679 to see if they have good suggestions.

@Julie-CAO
Copy link
Contributor Author

Julie-CAO commented Feb 7, 2025 •
edited
Loading

@Julie-CAO Hi Julie, yes , downloading assets from OSD require authentication, but still allowed. Can we add authentication related code instead?

I have not had a good idea about authentication. "curl" or "wget" with someone's username and API secrete? In this case, 3 more openqa settings are required and the automation would be complex.

Suggest to ask in https://suse.slack.com/archives/C02CANHLANP/p1737385075051679 to see if they have good suggestions.

Asked, https://suse.slack.com/archives/C02CANHLANP/p1738904207790159?thread_ts=1737385075.051679&cid=C02CANHLANP

@Julie-CAO
Copy link
Contributor Author

Marius gave two solutions:

  1. If you want to keep using LWP you would need to supply credentials via https://metacpan.org/pod/LWP::UserAgent#credentials. You can even make it work with our custom auth using which I did for asset downloads in openqa-clone-job in os-autoinst/openQA@773a5b1 (as those downloads are also using LWP).
  2. Can you specify this asset as mentioned on https://open.qa/docs/#_specifying_assets_required_by_a_job? Then the cache service would download it for you. (And if the file needs to be downloaded from the SUT you could use https://open.qa/api/testapi/#_autoinst_url.)

Comparing the above methods, I am thinking of option 2 by using ASSET_N ..., but part of the filename is runtime created, I have to find out a feasible way...

@Julie-CAO Julie-CAO changed the title Don't check URL on OSD repo before downloading guest assets as token is required [WIP]Don't check URL on OSD repo before downloading guest assets as token is required Feb 8, 2025
@Julie-CAO Julie-CAO changed the title [WIP]Don't check URL on OSD repo before downloading guest assets as token is required Use ASSET_n to download guest assets for virt tests Feb 10, 2025
@Julie-CAO Julie-CAO force-pushed the julie/no_check_download branch from 8758d29 to 1169d69 Compare February 10, 2025 04:44
@Julie-CAO
Copy link
Contributor Author

Julie-CAO commented Feb 10, 2025 •
edited
Loading

Hi, @alice-suse @waynechen55 @guoxuguang @nanzhg and others, A new solution is submitted, welcome review again. Verification is updated in PR description, more runs are on the way.

@Julie-CAO Julie-CAO force-pushed the julie/no_check_download branch from 1169d69 to 6b9b6e2 Compare February 10, 2025 13:12
@Julie-CAO Julie-CAO force-pushed the julie/no_check_download branch from 6b9b6e2 to f69cc65 Compare February 11, 2025 01:31
@Julie-CAO
Copy link
Contributor Author

Verification is done and updated in PR description. PR is ready.

@alice-suse
Copy link
Contributor

LGTM

@waynechen55
Copy link
Contributor

You just changed from using cp mounted_dir local_dir to curl ? @Julie-CAO

@Julie-CAO
Copy link
Contributor Author

You just changed from using cp mounted_dir local_dir to curl ? @Julie-CAO

Partly correct.

  • changed from cp mounted_dir local_dir , yes, exactly. Not use mount any more.
  • to curl, yes but not just curl files from OSD repo, but curl files from worker service. those files were downloaded at the beginning of the job by ASSET_n.

@alice-suse
Copy link
Contributor

Seems no objection. Merge it so as to unblock public beta potential tests. Thanks Julie.

@alice-suse alice-suse merged commit 120f19e into os-autoinst:master Feb 13, 2025
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@Julie-CAO @RoyCai7 @alice-suse @waynechen55